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I. Real Party in Interest 

The inventors Atsushi Watanabe and Taro Arimatsu assigned all rights in the subject 
application to FANUC LTD. on April 24, 2000 according to the Assignnnent executed April 24, 
2000 and submitted for recordation on June 7, 2000. Therefore, the real party in interest is 
FANUC LTD. 

II. Related Appeals and Interferences 

There are no related appeals or interferences known to Appellants, Appellants' legal 
representatives or the Assignee, FANUC LTD., which will directly affect or be directly affected by 
or have a bearing on the Board's decision in the pending appealB9/14/e0B6 YPOLITEl 88888855 89546213 

81 FC:1488 jgg^, 

III. Status of Claims 

Claims 1-14 are pending in the application and claims 1-14 stand rejected under 35 USC 
§ 103(a). 



IV. status of Amendments 

No Amendment was filed in response to the March 1 3, 2006 Office which stated that the 
Amendment filed by Certificate of Mailing on January 9, 2006 and received by the U.S. Patent 
and Trademark Office on January 1 1 , 2006 was entered. 

V. Summary of Claimed Subject Matter 

The present invention is directed to using a robot for generating "teaching models" used 
to detect "an object regardless of three-dimensional variations in position and posture of the 
object" (page 1 , lines 8-10). This is accomplished by capturing images of a reference object, 
according to different directions, orientations, positions, arrangements, etc. (page 3, lines 7-12). 
For example, four teaching models are generated based on image data obtained for the 
workpiece W viewed from four directions, as illustrated in Fig. 4 (page 6, lines 11-13). Image 
processing device 30 (Fig. 6) calculates information regarding the relative position and posture 
of the camera to the workpiece when the image was taken (page 14, last 3 lines and page 15, 
lines 1-7). The position and posture information is stored in non-volatile memory 37 with the 
image data which are used to automatically determine orientations of objects (workpieces) with 
shapes similar or identical to the reference object (page 16, lines 2-4). 

Claim 1 recites a "teaching model generating device for image processing, in which a 
subject object has the same or substantially similar shape as that of a reference object" (claim 1 , 
lines 1-2). One example of this in the specification is image processing device 30 (Fig. 6), as 
briefly described at page 8, lines 10-11. 

Next, claim 1 recites 

an image processing system with which a current three-dimensional orientation of 
the subject object relative to an image pickup device is recognized by carrying out 
pattern matching processing of an image of the subject based on a plurality of 
pre-determined teaching models of the reference object 

(claim 1 , lines 3-6). For example, as described on pages 22-23, image processing device 30 

(Fig. 6) performs operation 403 in Fig. 10. 

Prior to the operations performed by the image processing system recited on lines 3-6 of 
claim 1, "an image-capture system ... [generates and stores] the plurality of teaching models on 
the basis of respective image data produced by taking images of said reference object from a 
plurality of directions" (claim 1, lines 7-9). As described at page 17, line 1 to page 18, line 14, 
image processing system 30 (Fig. 6) can be used to perform the operations recited as being 
performed by the image-capture system. 
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The operations performed by the image-capture system are further defined in claim 1 in 
that "one of the reference object and said image pickup device is fixed to a movable and 
positionable part of a robot or is grasped with a hand of the robot, and said robot is operated for 
positioning to a plurality of different image pickup positions and directions" (claim 1 , lines 9-1 2), 
In a first embodiment described on pages 6-15, the camera is moved by the robot, as illustrated 
in Fig. 1. In a second embodiment described on pages 16-18, the camera is fixed in position 
and the robot moves the object, as illustrated in Fig. 3. In the third embodiment described on 
pages 18-21 , both the object and the camera are mounted on the end of a robot arm and may 
be moved, as illustrated in Fig. 3. 

Claim 1 ends by reciting that "the image data respectively obtained at each of said 
different image pickup positions and direction information indicating the respective different 
direction, is stored as a teaching model" (claim 1 , last 3 lines). This is described for each of the 
first through third embodiments, including at page 16, lines 2-4. 

The preamble of claim 2 is identical to that of claim 1 and therefore also has an example 
described at page 8, lines 10-11. Similarly, lines 3-6 of claim 2 are the same as lines 3-6 of 
claim 1 and thus, the example described on pages 22 and 23 of image processing device 30 
(Fig. 6) performing operation 403 in Fig. 10 applies to these lines of claim 2. In addition, claim 2 
recites ""an image-capture system ... generating and storing the plurality of teaching models on 
the basis of respective image data produced by taking images of said reference object from a 
plurality of directions" (claim 2, lines 7-9) using the same words as claim 1 . Thus, the 
description of image processing system 30 (page 17, line 1 to page 18, line 14; Fig. 6) as an 
example of the image-capture system applies to claim 2 also. 

The details recited on lines 9-15 of how the image-capture system recited in claim 2 
operates are different than in claim 1 . Claim 2 recites "the reference object is fixed to a movable 
part of a first robot or is grasped with a hand of the first robot, and said image pickup device is 
fixed to a movable and positionable part of a second robot or is grasped with a hand of the 
second robot" (claim 2, lines 9-12) and thus, the third embodiment described on pages 18-21 
with reference to Fig. 3 applies to claim 2. In addition, "any one of or both of said first and 
second robots is operated for positioning to a plurality of different relative image pickup positions 
and directions, so that the image data respectively obtained at each of said different image 
pickup positions is stored as a teaching model" (claim 2, last 4 lines) and thus, the operations 
described with respect to the first and second embodiments on pages 6-1 8 with reference to 
Figs. 1 and 2 also apply to claim 2. 
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Claim 8 is directed to a "method for teaching model generation and image processing" 
(claim 8, line 1). Examples of teaching model generation methods for each of the first through 
third embodiments are illustrated in Figs. 7-9 which are described on pages 6-21. As noted 
above, an example of an image processing method is illustrated in Fig. 10 and described on 
pages 22-23. 

The first operation recited in claim 8 is "determining a current three-dimensional 
orientation of a subject object relative to an image pickup device by carrying out pattern 
matching processing of an image of the subject based on a plurality of predetermined teaching 
models of a reference object" (claim 8, lines 2-4). The description of operations 403-406 in Fig. 
10 at page 22, line 19 to page 23, line 3 is an example of the determining operation recited on 
lines 2-4 of claim 8. 

The second operation recited in claim 8 is "generating and storing the plurality of 
teaching models on the basis of respective image data produced by taking images of said 
reference object from a plurality of directions" (claim 8, lines 5-7) which is performed "in advance 
of the determining" (claim 8, line 5). As noted above with respect to the same words in claim 1 , 
examples of this operation are described on pages 6-21. 

The details of how the second operation of claim 8 is performed are recited on lines 9-14 
of claim 8, where "one of the reference object and said image pickup device is fixed to a 
movable and positionable part of a robot or is grasped with a hand of the robot" (claim 8, lines 9- 
11) and "said robot is operated for positioning to a plurality of different image pickup positions 
and directions" (claim 8, lines 11-12) as described in the first through third embodiments on 
pages 6-21 and illustrated in Figs. 1-3, as discussed above with respect to claim 1 . Finally, "the 
image data respectively obtained at each of said different image pickup positions and direction 
information indicating the respective different direction, is stored as a teaching model" (claim 8, 
last 3 lines), as described at page 16, lines 2-4 for the first embodiment, page 18, lines 10-14 
with reference to blocks 204-208 in Fig. 8 for the second embodiment and page 21 , last 5 lines 
with reference to blocks 305-310 in Fig. 9 for the third embodiment. 

Claim 9 is directed to a "method of automatic orientation recognition" (claim 9, line 1). All 
of the inventive methods described in the specification are examples of "automatic orientation 
recognition". 

Examples of the first operation recited in claim 9, "generating and storing a set of images 
of different relative orientations of a subject" are described in each of the first through third 
embodiments on pages 6-21 , as discussed above. The further details recited as "the images 
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having been captured by a plurality of robotic operations corresponding to the different relative 
orientations of the subject, and associating with each inriage information indicating its respective 
relative orientation of the subject" (claim 9, lines 2-5) apply to all of the first through third 
embodiments where one or both of the camera and the reference object are moved by a robot 
arm. 

An example of the second operation of claim 9, "capturing a current image of a work- 
piece that has an unknown orientation relative to an image pickup device on the robot before the 
robot has come into contact with the workpiece, where the workpiece has a shape substantially 
similar to the shape of the subject" (claim 9, lines 6-9) which is performed "from a known current 
orientation of a robot" (claim 9, line 6), is described at page 22, lines 15-18 with reference to 
operation 402 in Fig. 10. 

An example of the third operation of claim 9, "using pattern matching to match one of the 
stored images with the current image" (claim 9, lines 10-11) is described at page 22, line 19 to 
page 23, line 10 with reference to operations 403-407 in Fig. 10. 

An example of the final operation of claim 9, "determining the orientation of the work- 
piece relative to the image pickup device on the robot based on the relative orientation informa- 
tion associated with the matched stored image, and also based on the known current orientation 
of the robot" (claim 9, last 3 lines) "after the pattern matching, and before the robot has come 
into contact with the workpiece" (claim 9, lines 12-13) is described in the paragraph spanning 
pages 23 and 24 with reference to operation 409 in Fig. 10. 

Like claim 9, claim 12 is directed to a "method of automatic orientation recognition" claim 
12, line 1) and therefore, all of the embodiments are examples of the method recited in claim 12. 

The first operation recited in claim 12, 

generating and storing a set of images of different relative arrangements of a 
subject, the images having been captured by a plurality of robotic operations 
corresponding to the different relative arrangements and associating with each 
image information indicating its respective relative arrangement of the subject 

(claim 12, lines 4-5) is consistent with how each of the first through third embodiments are 

described as obtaining image data of a reference object. Thus, the descriptions of the 

embodiments on pages 6-21 which are discussed above, apply to claim 12. 

An example of the second operation recited in claim 12, "with a known current 
arrangement of an image pickup device on a robot, capturing a current image of a workpiece 
with an unknown current arrangement relative to the robot, where the workpiece has a shape 
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substantially similar to the shape of the subject" (claim 12, lines 6-8) is described at page 22, 
lines 15-18 with respect to operation 402 in Fig. 10. 

An example of the third operation recited in claim 12, "using pattern matching to match 
one of the stored images with the current image" (claim 12, line 9) is described at page 22, line 
19 to page 23, line 10 with reference to operations 403-407 in Fig. 10. 

An example of the final operation recited in claim 12, "determining the current orientation 
of the workpiece relative to the image pickup device on the robot based on the relative arrange- 
ment information associated with the matched stored image, and also based on the known 
current arrangement of the image pickup device on the robot" (claim 1 2, last 4 lines) is described 
in the paragraph spanning pages 23 and 24 with reference to operation 409 in Fig. 10. 

The preamble of claim 1 3 does not include any details of the invention. An example of 
the first operation recited in claim 13, "robotically taking images of a subject with different three- 
dimensional subject-camera arrangements that vary in three dimensions, and associating with 
each image or data thereof information indicating its subject-camera arrangement" are described 
in each of the first through third embodiments on pages 6-21 , as discussed above. 

An example of the second operation recited in claim 13, "taking a current image of a 
workpiece shaped like the subject" (claim 13, line 5) is described at page 22, lines 15-18 with 
reference to operation 402 in Fig. 10. 

An example of the third operation recited in claim 13, "determining a current workpiece- 
camera orientation by matching one of the images or data thereof with the current image" (claim 
13, lines 7-8) "before picking up the workpiece" (claim 13, lines 5-6) is described at page 22, line 
19 to page 23, line 10 with reference to operations 403-407 in Fig. 10. 

An example of the final operation recited in claim 13, "using predetermined subject- 
camera arrangement information of the matched image to determine the three-dimensional 
orientation of the workpiece relative to the camera" (claim 13, last 3 lines) is described in the 
paragraph spanning pages 23 and 24 with reference to operation 409 in Fig. 10. 

Claim 14 is directed to at "least one computer-readable medium storing instructions that 
when executed control a robot apparatus to perform a method" (claim 14, lines 1-2). As stated 
in the sentence spanning pages 6 and 7 of the specification, an "operation program and its 
related set data of the robot taught according to an application are stored in the non-volatile 
memory of the memory 2" illustrated in Fig. 5. 
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An example of the first operation of the method recited in claim 14 as being performed 
when the instructions in the computer-readable medium are executed, "robotically taking images 
of a subject with a camera in different subject-camera arrangements varying in three 
dimensions" is described in each of the first through third embodiments on pages 6-21, as 
discussed above. 

An example of the second operation of the method recited in claim 14 as being 
performed when the instructions in the computer-readable medium are executed, "associating 
with each image, information indicating an obtaining subject-camera arrangement used in 
obtaining the image" (claim 14, lines 5-6) described, for example, in the paragraph spanning 
pages 23 and 24 with reference to operation 409 in Fig. 10. 

An example of the third operation of the method recited in claim 14 as being performed 
when the instructions in the computer-readable medium are executed, "after said associating of 
the images, taking a current image of a workpiece shaped like the subject" (claim 14, lines 7-8) 
is described at page 22, lines 15-18 with reference to operation 402 in Fig. 10. 

An example of the fourth operation of the method recited in claim 14 as being performed 
when the instructions in the computer-readable medium are executed, "determining a current 
workpiece-camera orientation by matching one of the images with the current image and using 
the information indicating the obtaining subject-camera arrangement associated with the one of 
the images" (claim 14, lines 9-11) is described at page 22, line 19 to page 23, line 10 with 
reference to operations 403-407 in Fig. 10. 

An example of the final operation of the method recited in claim 14 as being performed 
when the instructions in the computer-readable medium are executed, "picking up the workpiece 
after said determining" is described at page 24, lines 3-6 with reference to the operation 410 in 
Fig. 10. 

VI. Grounds of Rejection to be Reviewed on Appeal 

In the final Office Action dated March 13, 2006, the Examiner rejected claims 1, 3, 4, and 
8-14 were rejected under 35 U.S.C. § 103 as being unpatentable over U.S. Patent 4,462,046 to 
Spight in view of U.S. Patent 5,745,387 to Corby. Jr. et al. : claims 2, 5 and 6 were rejected as 
unpatentable over Spight in view of Corbv. Jr. et al. and further in view of U.S. Patent 4,504,970 
to Werth et al. : and claim 7 was rejected as unpatentable over Spight in view of Corby, Jr. et al. 
and Werth et al. and further in view of U.S. Patent 4,61 1 ,292 to Ninomiya et al. At issue is 
whether Spight in view of Corby. Jr. et al. , Werth et al. and Ninomiya et al. teach or suggest all of 
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the limitations recited in the claims and what motivation there is to combine the teachings of 
these references, particularly Spight and Corbv. Jr. et al. . 

VII. Argument 

In item 5 on pages 3-8 of the March 13, 2006 Office Action, claims 1, 3, 4, and 8-14 were 
rejected under 35 U.S.C. § 103 as being unpatentable over Spight in view of Corbv. Jr. et al. In 
rejecting claims 1 and 8, it was asserted that Spight "discloses that in order to produce real-time 
identification and determination of location and orientation of parts (object), a viewed representa- 
tion of the object is compared (pattern matching) with a stored representation of the desired 
objects" (March 13, 2006 Office Action, page 3, last 3 lines and page 4, line 1 , original emphasis 
removed) and from this "disclosure by Spight, it is clear that the stored image data is nothing but 
a plurality of teaching models" (March 13, 2006 Office Action, page 4, lines 2-3). 

In addition, it was acknowledged that Spight does not teach that "said robot is operated 
for positioning to a plurality of different image pickup positions and directions" (e.g., claim 1 , lines 
11-12) to obtain "the image data ... stored as a teaching model" (e.g., claim 1 , last 3 lines), but it 
was asserted that Corbv, Jr. et al. can be combined with Spight to provide this feature of the 
claimed invention. It is submitted that insufficient evidence has been cited to support a finding 
that one of ordinary skill in the art would find it obvious to combine the teachings of Spight and 
Corby. Jr. et al. 

No statement has been found in the March 13, 2006 Office Action regarding what would 
motivate one of ordinary skill in the art to combine the teachings of Spight and Corbv. Jr. et al. , 
let alone, "[ejxplicit findings on motivation or suggestion to select the claimed invention ... to 
support a 35 U.S.C. 103 ground of rejection" as required in MPEP 2144,08. It is well 
established by case law that "[t]he factual inquiry whether to combine references ... must be 
based on objective evidence of record. ... [The] factual question of motivation ... cannot be 
resolved on subjective belief and unknown authority." In re Lee, 277 R3d 1338, 1343-4, 61 
USPQ2d 1430 (Fed. Cir. 2002). 

The only statements that have been found in the March 13, 2006 Office Action that come 

close to suggesting a motivation to combine Spight and Corbv. Jr. et al. are that 

it would have been obvious to one of ordinary skill in the art at the time the inven- 
tion was made to fix an image pickup device to a movable and positionable part 
of a robot, operate the robot for positioning to a plurality of different image pickup 
positions and directions, and store direction information indicating the respective 
different direction[s] with the image data as a teaching model as taught by Corby 
in order to define the identity of the site imaged, when it was imaged, the view- 
point, the modality of the imager and description of values relating to the image ... 
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and ... to further enhance the object identification when [the] object is viewed at 
different directions or viewpoints, keeping in view the 3 dimensional structure of 
the object, in real-time and further both references are directed to robotic 
industrial automation 

(March 13, 2006 Office Action, page 5, last line to page 6, line 9). With the exception of the 
statement that "both references are directed to robotic industrial automation", all that seems to 
be described is what the Examiner asserted one of ordinary skill in the art would learn from 
Corbv. Jr. et al. No reason was provided for why this would motivate one of ordinary skill in the 
art to combine the teachings of Corbv. Jr. et al. with those of Spight . The mere fact that "both 
references are directed to robotic industrial automation" is insufficient. Examiners are required 
to "explain the reasons one of ordinary skill in the art would have been motivated to select the 
references." In re Lee, 61 USPQ2d at 1434. "Obviousness can not be established by combining 
the teachings of the prior art to produce the claimed invention, absent some teaching, sugges- 
tion or incentive supporting the combination." In re Gorman, 933 F2d 982, 18 USPQ2d 1885, 
1888 (Fed. Cir. 1991) quoting In re Bond, 910 F2d 831, 834, 15 USPQ2d 1566, 1568 (Fed. Cir. 
1990) which in turn quoted Carella v. Starlight Archery and Pro Line Co., 804 R2d 135, 140, 231 
USPQ 644, 647 (Fed. Cir. 1986). 

There is nothing on the record to explain what advantage would be gained by combining 
Corbv, Jr. et al. with Spight . The mere fact that Corbv, Jr. et al. discloses a conventional posi- 
tioning of a camera by a robot arm and the claimed invention can also use such an arrangement 
does not provide a reason for modifying Spight . Rather, the rejection of the claims over the 
combination of Spight and Corbv. Jr. et al. suggests that impermissible hindsight has been used 
to find a reference (Corbv, Jr. et al.) which discloses features of the invention that are missing 
from Spight without providing a reason for a person of ordinary skill in the art to combine the 
teachings of these references. 

Item 10 on pages 11-12 of the March 13, 2006 Office Action responded to the arguments 
in the January 9, 2006 Amendment that the Background of the Invention section in Corbv, Jr et 
aL teaches away from using Corbv. Jr. et al. to modify Spight or that Corbv, Jr. et al. is relevant to 
the invention, since neither Spight nor the invention are directed to operations in "environments 
which are inaccessible or very hazardous for humans" or "to determine the rate of deterioration" 
as in Corbv. Jr. et al. In this response, it was asserted that the "Examiner does not rely on Corby 
to provide teachings on the hazardous environments for humans" (March 13, 2006 Office Action, 
page 12, lines 4-5). It is submitted that why the Examiner is relying on Corbv, Jr. et al. is not the 
test for whether one of ordinary skill in the art would find it obvious to combine Corbv, Jr. et al. 
and Spight . The issue is whether the entirety of the teachings in Corbv. Jr. et al. and Spight 
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would lead one of ordinary skill in the art to combine Corbv, Jr. et al. and Spight . It is submitted 
that the fact that Corby, Jr. et al. is concerned with hazardous environments for humans and 
Spight is not is relevant to determining the issue of obviousness to combine their teachings. 

Further, contrary to the implication at page 12, lines 15-17 of the March 13, 2006 Office 
Action that Corbv. Jr. et al. discloses "pattern matching" by a processor, as that term is used in 
the application, as discussed in the January 9, 2006 Amendment there is no suggestion in 
Corby, Jr. et al. of storing a plurality of views for the purpose of identifying anything. The 
archive and comparison device taught by Corbv, Jr. et al. is used for determining deterioration at 
a given inspection site, not for identification. No description has been cited or found in Corby, Jr. 
et al. regarding how an operator determines what file should be used for comparison. 
Presumably the typical technique of using a naming convention for the files would be used, since 
other textual data can be input via text input device 57 (see column 6, lines 41-51). There is not 
the slightest suggestion that the site where deterioration is occurring is identified "by carrying out 
pattern matching processing of an image" (claim 1, lines 5-6), either automatically, or by an 
operator. Rather, Corbv. Jr. et al. merely describes recording images of a site from various 
viewpoints and providing the images for comparison by a human operator. 

The lack of similarity in how the images are used in Corbv. Jr. et al. and Spight is an 
additional reason why one of ordinary skill in the art would not find it obvious to combine Corbv, 
Jr. et al. and Spight . The automatic processing of images taught by Spight is significantly 
different from the human comparison of images taught by Corbv, Jr. et al. For all of the above 
reasons, it is submitted that claims 1 , 8, 9 and 1 2-1 4, as well as claims 3,4,10 and 1 1 which 
depend therefrom, patentably distinguish over the applied art. 

In item 6 on pages 8-10 of the March 13, 2006 Office Action, claims 2, 5 and 6 were 
rejected under 35 U.S.C. § 103 as being unpatentable over Spight in view of Corbv. Jr. et al. and 
further in view of Werth et al. In item 7 on page 10 of the Office Action, claim 7 was rejected as 
unpatentable over Spight in view of Corbv. Jr. et al. and Werth et al. and further in view of 
Ninomiva et al. Nothing has been cited or found in Werth et al. or Ninomiva et al. suggesting 
modification of the combination of Spight and Corbv, Jr. et al. to overcome the deficiencies 
discussed above. Therefore, it is submitted that claims 2 and 5-7 patentably distinguish over 
Spight in view of Corbv, Jr. et a!. , Werth et al. and Ninomiva et al. for at least the reasons set 
forth above with respect to claim 1 from which they depend. 
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Summary of Arguments 

For the reasons set forth above and in the Response filed April 14, 2005 and the 
Amendment filed January 9, 2006, it is submitted that claims 1-14 patentably distinguish over 
Spight , Corbv, Jr. et al. . Werth et al. and Ninomiva et al. Thus, it is respectfully submitted that 
the Examiner's final rejection of the claims is without support and, therefore, erroneous. 
Accordingly, the Board of Patent Appeals and Interferences is respectfully urged to so find and 
to reverse the Examiner's final rejection. 

The required fee in the amount of $500.00 is attached. If any additional fees are 
required, please charge same to our Deposit Account No. 19-3935. 



Respectfully submitted 



STAAS & HALSEY LLP 





Richard A. Gollhofer 
Registration No. 31,106 



1201 New York Avenue, NW, Suite 700 
Washington, D.C. 20005 
Telephone: (202)434-1500 
Facsimile: (202) 434-1501 
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VIII. Claims Appendix 

1. A teaching model generating device for image processing, in which a subject object 
has the same or substantially similar shape as that of a reference object, the device comprising: 

an image processing system with which a current three-dimensional orientation of the 
subject object relative to an image pickup device is recognized by carrying out pattern matching 
processing of an image of the subject based on a plurality of pre-determined teaching models of 
the reference object; and 

an image-capture system, in advance of the recognizing, generating and storing the 
plurality of teaching models on the basis of respective image data produced by taking images of 
said reference object from a plurality of directions, wherein one of the reference object and said 
image pickup device is fixed to a movable and positionable part of a robot or is grasped with a 
hand of the robot, and said robot is operated for positioning to a plurality of different image 
pickup positions and directions, so that the image data respectively obtained at each of said 
different image pickup positions and direction information indicating the respective different 
direction, is stored as a teaching model. 

2. A teaching model generating device for image processing, in which a subject object 
has the same or substantially similar shape as that of a reference object, the device comprising: 

an image processing system with which a current three-dimensional orientation of the 
subject object relative to an image pickup device is recognized by carrying out pattern matching 
processing of an image of the subject based on a plurality of pre-determined teaching models of 
the reference object; and 

an image-capture system, in advance of the recognizing, generating and storing the 
plurality of teaching models on the basis of respective image data produced by taking images of 
said reference object from a plurality of directions, wherein the reference object is fixed to a 
movable part of a first robot or is grasped with a hand of the first robot, and said image pickup 
device is fixed to a movable and positionable part of a second robot or is grasped with a hand of 
the second robot, and any one of or both of said first and second robots is operated for 
positioning to a plurality of different relative image pickup positions and directions, so that the 
image data respectively obtained at each of said different image pickup positions is stored as a 
teaching model. 

3. A teaching model generating device according to claim 1, wherein said teaching 
model is a part of the image data of the reference object. 
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4. A teaching model generating device according to claim 1, wherein said teaching 
model comprises data obtained by performing image processing on the image data of the 
reference object. 

5. A teaching model generating device according to claim 2, wherein said teaching 
model is generated for every direction in which said image pickup device took the image of said 
reference object and said teaching model is stored in association with information on the 
direction. 

6. A teaching model generating device according to claim 2, wherein said image pickup 
device is a camera. 

7. A teaching model generating device according to claim 2, wherein said image pickup 
device is a three-dimensional visual sensor that measures a distance between the image pickup 
device and a plurality of points on the object. 

8. A method for teaching model generation and image processing, comprising: 
determining a current three-dimensional orientation of a subject object relative to an 

image pickup device by carrying out pattern matching processing of an image of the subject 
based on a plurality of predetermined teaching models of a reference object; and 

in advance of the determining, generating and storing the plurality of teaching models on 
the basis of respective image data produced by taking images of said reference object from a 
plurality of directions, wherein one of the reference object and said image pickup device is fixed 
to a movable and positionable part of a robot or is grasped with a hand of the robot, and said 
robot is operated for positioning to a plurality of different image pickup positions and directions, 
so that the image data respectively obtained at each of said image pickup positions and direction 
information indicating the respective different direction, is stored as a teaching model. 

9. A method of automatic orientation recognition, comprising: 

generating and storing a set of images of different relative orientations of a subject, the 
images having been captured by a plurality of robotic operations corresponding to the different 
relative orientations of the subject, and associating with each image information indicating its 
respective relative orientation of the subject; 
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after the generating and storing, from a known current orientation of a robot, capturing a 
current image of a workpiece that has an unknown orientation relative to an image pickup device 
on the robot before the robot has come into contact with the workpiece, where the workpiece 
has a shape substantially similar to the shape of the subject; 

after the capturing, using pattern matching to match one of the stored images with the 
current image; and 

after the pattern matching, and before the robot has come into contact with the 
workpiece, determining the orientation of the workpiece relative to the image pickup device on 
the robot based on the relative orientation information associated with the matched stored 
image, and also based on the known current orientation of the robot. 

10. A method according to claim 9, further comprising automatically maneuvering the 
robot to the workpiece based at least on the determined orientation of the workpiece relative to 
the robot. 

11 . A method according to claim 9, wherein the generating and storing is performed for a 
plurality of differently shaped subjects, wherein the current image includes a plurality of 
differently shaped workpieces, and wherein the pattern matching further comprises identifying 
the workpiece from among the plurality of differently shaped workpieces using the images and 
orientation information of the plurality of differently shaped subjects. 

12. A method of automatic orientation recognition, comprising: 

generating and storing a set of images of different relative arrangements of a subject, the 
images having been captured by a plurality of robotic operations corresponding to the different 
relative arrangements, and associating with each image information indicating its respective 
relative arrangement of the subject; 

with a known current arrangement of an image pickup device on a robot, capturing a 
current image of a workpiece with an unknown current arrangement relative to the robot, where 
the workpiece has a shape substantially similar to the shape of the subject; 

using pattern matching to match one of the stored images with the current image; and 

determining the current orientation of the workpiece relative to the image pickup device 
on the robot based on the relative arrangement information associated with the matched stored 
image, and also based on the known current arrangement of the image pickup device on the 
robot. 
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13. A method, comprising: 

robotically taking images of a subject with different three-dimensional subject-camera 
arrangements that vary in three dimensions, and associating with each image or data thereof 
information indicating its subject-camera arrangement; then 

taking a current image of a workpiece shaped like the subject; and then before picking up 
the workpiece 

determining a current workpiece-camera orientation by matching one of the images or 
data thereof with the current image, and using predetermined subject-camera arrangement 
information of the matched image to determine the three-dimensional orientation of the 
workpiece relative to the camera. 

14. At least one computer-readable medium storing instructions that when executed 
control a robot apparatus to perform a method comprising: 

robotically taking images of a subject with a camera in different subject-camera 
arrangements varying in three dimensions; 

associating with each image, information indicating an obtaining subject-camera 
arrangement used in obtaining the image; 

after said associating of the images, taking a current image of a workpiece shaped like 
the subject; 

determining a current workpiece-camera orientation by matching one of the images with 
the current image and using the information indicating the obtaining subject-camera arrange- 
ment associated with the one of the images; and 

picking up the workpiece after said determining. 
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IX. Evidence Appendix 

(None) 
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Related Proceedings Append 

(None) 



